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Abstract 

This  paper  presents  a  method  for  in¬ 
crementally  constructing  CEO  succession 
timelines  via  a  cascade  of  minimally  su¬ 
pervised  infomiation  synthesis  compo¬ 
nents.  Results  from  multiple  documents 
are  combined  using  a  CRF  fusion  method. 
The  system  demonstrates  that  with  min¬ 
imal  training  on  the  target  domain,  the 
presence  of  redundant  infomiation  allows 
for  the  synthesis  of  networks  of  interre¬ 
lated  facts  from  text. 
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Figure  1 :  The  goal  of  the  system  is  a  database  filled 
with  time -bounded  CEO  tenure  facts.  Separate  in¬ 
fomiation  synthesis  components  fill  different  fields 
of  the  database  incrementally. 


1  Introduction 

Single  document  infomiation  extraction  of  named 
entities  and  relationships  has  received  much  atten¬ 
tion  (e.g.  MUC  and  ACE1).  Relatively  less  explored 
is  multi-document  information  synthesis,  where  in¬ 
fomiation  contained  in  separate  documents  within  a 
large  corpus  is  automatically  extracted  and  fused  to 
fonn  networks  of  related  facts.  Timelines  are  an  im¬ 
portant  example  of  such  networks  as  they  use  tem¬ 
poral  infomiation  to  resolve  ambiguities  in  extracted 
facts. 

This  paper  presents  a  method  for  synthesizing 
CEO  succession  timelines  from  multiple  documents 
without  annotated  data.  The  core  of  the  method  is 
an  infomiation  synthesis  component  which  performs 
document  retrieval  (Section  2.1),  sentence  extraction 
(Section  2.2),  and  cross-document  fusion  (Section 
2.3). 

1http://www.itl.  nist.gov/iaui/894.02/related_projects/muc/ 
and  http://www.itl.nist.gov/iad/894.01/tests/ace/. 


A  pipeline  of  these  synthesis  components  is  used 
to  build  a  database  of  time -bounded  CEO  succes¬ 
sion  facts  (Section  3).  The  pipeline  produces  a  list 
of  candidate  CEOs  (Section  3.1),  direct  transitions 
between  CEOs  (Section  3.2),  CEO  tenure  midpoints 
(Section  3.3),  and  CEO  start  and  end  years  (Section 
3.4).  System  performance  is  given  for  each  compo¬ 
nent  as  well  as  for  the  overall  database  (Section  5). 

2  Information  Synthesis  via  Retrieval, 
Extraction  and  Fusion 

The  goal  of  the  system  is  to  fill  a  relational  database 
(e.g.  Figure  1)  in  which  each  database  record  cor¬ 
responds  to  a  particular  time -bounded  fact.  In  the 
filled  database,  this  collection  of  records  constitutes 
a  timeline.  The  system  works  incrementally  to  build 
up  the  entire  database,  leveraging  a  partially  filled 
database  at  each  step  in  the  process.  The  filled  fields 
in  the  database  will  be  referred  to  as  the  base  fields, 
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and  the  current  cell  being  tilled  as  the  target  field2. 
Each  target  field  is  also  assigned  a  target  type  (e.g. 
PERSON)  and  words  which  belong  to  this  type  are 
marked  during  pre-processing  stages.  To  find  each 
target  field,  there  is  an  information  synthesis  compo¬ 
nent  composed  of  three  stages:  document  retrieval, 
sentence  extraction  and  cross-document  fusion. 


2.1  Document  Retrieval 

While  very  large  corpora  frequently  contain  redun¬ 
dant  information,  their  use  prohibits  exhaustive  ap¬ 
plication  of  complex  information  extraction  meth¬ 
ods.  Evaluating  a  CRF  on  all  documents  on  the 
Internet  is  infeasible.  During  document  retrieval,  a 
sub-corpus  is  selected  from  a  larger  corpus,  allowing 
for  deep  processing  of  the  documents  that  are  most 
likely  to  contain  the  information  of  interest.  The 
process  of  document  retrieval  is  as  follows:  From 
the  base  fields,  a  base  query  is  formulated.  The 
query  is  issued  to  Google,  which  returns  a  ranked  list 
of  documents.  The  documents  on  this  list  are  down¬ 
loaded  and  preprocessed  in  series  using  the  Penn  To- 
kenizer  (MacIntyre,  1995),  a  part-of-speech  tagger 
(Florian  and  Ngai,  2001),  and  a  Named  Entity  tag¬ 
ger3. 

2.2  Sentence  Extraction 

Once  a  set  of  documents  has  been  gathered,  sen¬ 
tences  which  contain  the  base  fields  and  a  candidate 
target4  are  selected,  and  a  sentence  extraction  sys¬ 
tem  is  applied  over  the  sentences.  This  paper  uses 
Linear  Chain  Conditional  Random  Fields  (CRFs) 
(Lafferty  et  ah,  2001),  an  undirected  conditional 
graphical  model,  to  extract  facts  from  sentences.  A 
CRF  yields  a  distribution  P(Y\X)  of  hidden  labels 
Y  for  an  observation  sequence  X : 


P{Y\X)  =  —  exp((^^  Xkfk(yt-i,yt,x,t) 


t=  1  k 


sequence,  and  the  linear  chain  index  t,  and  A/,  is  the 
weight  for  a  feature  //... 

During  training,  a  base  query  is  issued  to  Google 
for  each  training  record,  and  the  resulting  corpus  is 
then  automatically  annotated  using  the  training  base 
and  target  fields.  This  annotation  is  then  used  to  train 
a  CRF.  The  automatic  annotation  step  marks  all  text 
which  matches  a  base  field  or  a  target  field.  The 
models  are  trained  with  the  assumption  that  base 
fields  have  been  correctly  filled.  During  training,  in 
sentences  where  the  target  field  isn’t  found,  items  of 
the  target  field  type  are  labeled  as  spurious  targets. 

2.3  Cross-Document  Fusion 

After  extraction  is  performed,  it  is  necessary  to  fuse 
the  targets  to  arrive  at  a  consensus  answer.  Prior 
methods  which  use  a  CRF  extractor  for  sentence 
level  extraction  have  used  Viterbi  frequency  fusion 
(VFF),  whereby  the  system  chooses  the  fact  mostly 
frequently  extracted  by  the  CRF  Viterbi  labeling  se¬ 
quence  from  all  sentences  in  the  corpus  (Mann  and 
Yarowsky,  2005). 

This  paper  proposes  using  field  confidence  to  fuse 
extracted  facts.  The  field  confidence  is  the  probabil¬ 
ity  of  a  word  being  assigned  a  certain  label,  summed 
over  all  other  possible  labels  for  the  other  words.  If 
yr  is  the  label  that  indicates  that  a  given  word  Xrn  is 
the  target  for  a  relationship  r,  then: 

P(r(Xm)\X)  =  P(Ym  =  y\X)  =  £  P(Y'\X) 

Y':Ym=y 

The  field  confidence  can  be  efficiently  computed  us¬ 
ing  the  Constrained  Forward-Backward  algorithm 
(Culotta  and  McCall um,  2004).  From  the  field  confi¬ 
dence  the  maximum  field  confidence  score  (FCM) 
for  a  given  target  over  all  sentences  s  can  be  com¬ 
puted  : 


CFCM(Xm)  =  max  P(r(X^)\Xs) 

S 


Where  Z  is  a  normalization  constant,  t  is  a  index 
into  the  linear  chain,  fy-  is  a  binary  feature  function 
over  the  label  yt  and  its  predecessor,  the  observation 

2In  some  cases,  the  target  field  is  a  derived  field  not  explic¬ 
itly  present  in  the  database. 

3The  Named  Entity  tagger  is  a  CRF  which  is  trained  front 
the  MUC-7  training  corpora  and  achieves  around  81%  F- 
Measure  on  the  MUC-7  test  corpus. 

4Candidate  targets  are  identified  by  target  type. 


Alternatively,  a  field  confidence  fusion  (FCF)  score 
can  be  taken  as  sum  over  all  sentences  of  the  field 
confidence  probability: 

CFCF(X.m)  =  J2nr(Xsm)\Xs) 

S 

The  fusion  method  described  above  is  used  for  fu¬ 
sion  of  facts  from  one  extractor.  Sections  3.3  and  3.5 


introduce  additional  methods  for  fusing  facts  from 
multiple  extractors. 

Cross-document  fusion  often  requires  some  form 
of  fact  normalization,  as  the  same  target  can  be  ex¬ 
pressed  in  a  variety  of  ways.  For  CEO  succession 
timeline  construction,  the  targets  which  exhibit  the 
most  variation  are  names.  A  simple  name  matcher 
was  developed  to  perform  this  normalization,  which 
uses  a  set  of  name-nickname  pairs5  to  merge  first 
names  and  optionally  drops  middle-initials. 

3  Timeline  Construction 

For  the  problem  of  timeline  construction,  the  infor¬ 
mation  synthesis  component  described  in  the  previ¬ 
ous  section  is  applied  a  number  of  times  with  differ¬ 
ent  training  data  to  synthesize  different  fields  in  the 
database  (Figure  2).  The  pipeline  is  as  follows: 

1 .  Given  a  company,  the  system  generates  a  set  of 
CEOs  for  that  company  and  the  top  candidate 
is  picked  (Section  3.1). 
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Figure  2:  A  series  of  infomiation  synthesis  compo¬ 
nents  builds  up  a  CEO  succession  timeline.  Later 
steps  in  the  pipeline  are  less  reliable  and  correspond 
to  more  fine-grained  infomiation. 


3.2  Direct  Succession 


2.  The  direct  succession  model  selects  an  adjacent 
CEO  from  the  remaining  candidates.  (Section 
3.2). 

3.  For  each  CEO  in  the  pair,  an  estimated  tenure 
midpoint  (Section  3.3)  and  start  and  end  tenure 
years  (Section  3.4)  are  found.  The  pair  or¬ 
der  and  the  member  start  and  end  years  are 
combined  to  arrive  at  a  transition  year  estimate 
(Section  3.5). 

3.1  CEO  Name 

The  first  step  in  the  pipeline  finds  a  list  of  candidate 
CEOs  which  are  used  for  the  remainder  of  the  run. 
The  base  fields  for  the  extractor  are  the  company 
name  and  the  title.  The  base  query  (e.g.  “Boeing 
CEO”)  is  issued  to  Google  and  the  top  1000  docu¬ 
ments  are  returned.  The  documents  are  marked  with 
occurrences  of  the  base  field,  the  sentence  extrac¬ 
tor  is  applied  over  the  sentences,  and  the  extracted 
CEOs  are  then  fused  (Section  2.3).  The  system 
chooses  the  first  CEO  from  the  ranked  list,  and  for 
further  CEOs,  the  direct  succession  model  is  used. 

Collected  by  P.  Driscoll. 


A  second  CEO  is  chosen  from  the  ranked  list  using  a 
direct  succession  model,  which  finds  sentences  that 
explicitly  mention  a  succession  relationship.  The 
model  is  trained  with  two  CEOs  as  a  base,  where 
the  label  sequence  declares  them  to  be  in  a  partic¬ 
ular  order,  and  the  target  of  extraction  is  the  rel¬ 
ative  order  of  two  given  CEOs.  For  example,  for 
the  database  shown  in  Figure  1,  a  training  sentence 
might  be  marked: 

When Ipre  Alfred  Iprev  Sloan  Iprev  re¬ 
tired///;/  Jint  Charles Inext  Wilson//ie\7 
replaced/sw/  him  isuf  .Isuf 

where  the  labels  are  pre fix,  previous  in  relationship, 
interstitial,  next  in  relationship,  and  suffix. 

To  choose  a  successor  or  predecessor  for  the  top 
ranked  CEO,  each  candidate  ordered  pair  is  sepa¬ 
rately  searched  for,  and  extraction  and  fusion  are 
performed.  Each  base  query  is  a  pair  of  names  (e.g. 
“Jeffrey-Immelt  Jack- Welch”),  and  100  documents 
from  the  Google  returned  list  are  returned  for  each 
pair. 

3.3  Tenure  Midpoint  Estimation 

For  tenure  midpoint  estimation,  the  base  is  a  CEO 
name  and  the  target  is  the  midpoint  of  the  CEO’s 


tenure  in  that  position.  For  this  component  and 
subsequent  components,  each  base  query  is  a  CEO 
name  and  company  (e.g.  “GE  Jack- Welch”),  and 
100  documents  are  retrieved.  The  tenure  midpoint 
is  estimated  by  extracting  years  for  which  the  CEO 
was  in  office  and  taking  a  weighted  sum  over  the 
list  of  years.  To  build  the  sentence  extractor  to  ex¬ 
tract  the  years  in  which  the  CEO  was  in  office,  these 
tenure  years  are  marked  in  the  training  corpus.  Al¬ 
though  the  confidence  estimates  for  particular  tenure 
years  are  often  noisy,  the  estimated  tenure  midpoints 
provide  a  second  source  of  information  about  the  rel¬ 
ative  ordering  between  two  people. 

3.4  Start/End  Tenure  Year 

For  start  and  end  tenure  years,  the  base  fields  are 
CEOs  and  the  targets  are  the  desired  year.  Start  and 
end  years  are  the  least  reliable  information  from  all 
of  the  different  methods,  because  they  do  not  appear 
frequently  in  text,  whereas  all  of  the  prior  methods 
are  able  to  use  corpus  redundancies  to  boost  accu¬ 
racy. 

3.5  Transition  Year  Estimation 

Underlying  data  dependencies  are  useful  in  increas¬ 
ing  the  accuracy  of  the  exact  start/end  years.  Know¬ 
ing  the  information  succeed(A,B),  then  it  must 
be  true  that  end(A,X)  and  start  (B,X).  Two 
methods  for  using  this  information  are  considered. 
In  the  first,  the  end  tenure  year  for  A  is  thrown  out 
and  replaced  bystart(B,x),as  start  year  predic¬ 
tion  is  known  to  be  higher  confidence.  In  the  second 
method,  a  linear  combination  is  used  to  provide  a 
new  estimate  of  the  transition  year.  Given  C'J' (X), 
the  confidence  for  the  end  year  of  the  predecessor 
A,  and  CB(X),  the  confidence  for  the  start  year  of 
the  successor  B,  and  the  estimate  for  C\B(X),  the 
confidence  for  a  transition  year  X  is: 

CTab{X)  =  Cf{X)  x  CSB(X) 

4  Example  Pipeline  Run 

An  example  of  the  pipeline  is  shown  for  the  com¬ 
pany  Gannett.  Table  1  shows  an  example  top  ten  list 
for  Gannett  CEO  extraction.  The  top  ten  list  con¬ 
tains  5/6  of  the  total  possible  Gannett  CEOs,  where 
incorrect  candidates  are  primarily  heads  of  other  di¬ 
visions  within  Gannett:.  The  top  choice,  Douglas 
McCorkindale  is  entered  into  the  database. 


Name 

Confidence 

Douglas  H.  McCorkindale 

0.181 

Craig  A.  Dubow 

0.092 

Allen  H.  Neuharth 

0.068 

Cecil  L.  Walker 

0.065 

John  J.  Curley 

0.046 

Frank  Gannett 

0.042 

Roger  L.  Ogden 

0.01 

Ken  Tonning 

0.006 

Craig  Moon 

0.006 

Mimi  Feller 

0.005 

Table  1 :  CEOs  extracted  for  Gannett.  Correct  CEOs 
in  bold.  Douglas  McCorkindale,  the  top  ranked  can¬ 
didate,  was  CEO  of  Gannett  from  2000-2005. 


Proposed  CEO  order  (A,B) 
X  =  McCorkindale 

Confidence 

X,  Craig  Dubow 

5.193 

X,  John  Curley 

4.460 

John  Curley,  X 

3.835 

Craig  Dubow,  X 

3.113 

Craig  Moon,  X 

1.034 

X,  Craig  Moon 

1.007 

X,  Cecil  Walker 

0.949 

Frank  Gannett,  X 

0.927 

Cecil  Walker,  X 

0.854 

X,  Frank  Gannett 

0.124 

Table  2:  FCF  scores  for  pairs  in  Direct  Succession 
Model.  Correct  relative  orderings  in  bold.  (Mc¬ 
Corkindale,  Dubow)  is  the  highest  ranking  proposed 
pair. 

The  system  then  picks  a  second  CEO  from  the  list 
using  the  direct  succession  model.  Table  2  lists  the 
field  confidence  fusion  scores  associated  with  vari¬ 
ous  ordered  pairs.  The  top  pair,  (Douglas  McCorkin¬ 
dale,  Craig  Dubow),  is  entered  into  the  database. 

The  tenure  years  for  McCorkindale  and  Dubow 
are  then  extracted  and  tenure  midpoints  are  esti¬ 
mated.  McCorkindale  (2000-2005)  has  an  estimated 
tenure  midpoint  of  1998.  Dubow  (2005-)  has  an 
estimated  tenure  midpoint  of  2001.  Given  these 
tenure  midpoints,  the  ordering  proposed  in  the  previ¬ 
ous  step  is  confirmed,  with  McCorkindale  preceding 
Dubow. 

The  start  and  end  years  for  McCorkindale  and 


Company 

Title 

Name 

Start 

End 

Gannett 

CEO 

John  Curley 

X 

2000 

Gannett 

CEO 

Douglas  McCorkindale 

2000 

2005 

Gannett 

CEO 

Craig  Dubow 

2005 

X 

Figure  3:  Graph  of  CEO  tenure  year  confidence  for 
Gannett.  The  three  most  recent  CEOs  have  been  cor¬ 
rectly  identified.  Below  the  graph  is  the  extracted 
database. 

Dubow  are  extracted  as  well.  The  estimated  span 
points  for  McCorkindale  are  (2000-2003),  with  the 
correct  year,  2005,  ranked  10th.  For  Dubow,  the  esti¬ 
mated  start  and  end  years  are  both  2005.  In  this  case 
the  re-estimated  transition  year  from  fusion  yields 
the  year  2005,  which  is  correct.  If  the  system  were  to 
perform  another  iteration,  CEO  John  Curley  would 
be  found,  producing  the  timeline  depicted  in  Figure 
3. 

5  Detailed  Experimental  Results 

The  database  used  for  the  experiments  reported  in 
the  following  section  took  a  sample  of  18  compa¬ 
nies  from  the  Fortune  500  list6.  For  each  company, 
the  author  used  the  Internet  to  find  ground  truth  of 
the  entire  CEO  history  for  the  company.  Of  the 
companies,  six  were  randomly  chosen  and  selected 
for  training  (Anheuser-Busch,  Hewlett-Packard, 
Fennar,  McGraw-Hill,  Pfizer,  and  Raytheon),  four 
were  used  as  a  development  set  (Boeing,  Heinz,  Sta¬ 
ples,  and  Textron),  and  eight  were  used  for  testing 
(Gannett,  General  Electric,  General  Motors,  Home- 
Depot,  IBM,  Kroger,  Sears,  and  UPS).  Altogether, 
there  were  98  database  records  for  the  companies, 
with  21  training  records,  16  development  records, 
and  61  testing  records. 

6http : // www . fortune . com/ for tune/ f ortune500 
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Figure  4:  CEO  mentions  over  time.  Much  more 
information  is  available  for  recent  CEOs  than  for 
CEOs  from  earlier  decades. 

One  peculiarity  of  the  data  is  that  CEOs  from  ear¬ 
lier  decades  have  very  few  mentions  on  the  web  as 
compared  with  more  recent  CEOs  (see  Figure  4). 
Many  of  the  missing  CEOs  in  the  CEO  name  ex¬ 
traction  step  never  appeared  on  the  web.  For  exam¬ 
ple,  for  Fowler  McConnell,  Sears  CEO  from  1958  to 
1960,  the  query  “Fowler-McConnell  Sears”  returns 
exactly  2  documents. 

5.1  CEO  Name 

The  precision  and  recall  of  the  returned  ranked  lists 
can  be  calculated  for  the  CEO  name  component. 
Figure  5  graphs  precision  and  recall  for  CEO  ex¬ 
traction  on  the  8  test  set  companies  using  the  Viterbi 
frequency  fusion  (VFF),  field  confidence  maximum 
(FCM),  and  field  confidence  fusion  (FCF).  Precision 
near  the  top  of  the  ranked  list  is  quite  high  -  more 
than  90%  of  the  returned  top  2  CEOs  are  correct. 
Further,  recall  never  reaches  more  than  70%.  As 
previously  mentioned,  this  is  primarily  due  to  sparse 
data  for  CEOs  in  earlier  decades,  before  periodicals 
were  published  widely  on  the  Internet.  Finally,  FCF 
performs  slightly  better  than  VFF  and  FCM,  partic¬ 
ularly  towards  the  top  of  the  ranked  list,  which  is  the 
most  crucial. 

5.2  Synthesis  of  Temporal  Information 

Once  the  CEO  name  list  has  been  extracted  and  the 
top  CEO  candidate  selected  as  a  future  base  fact,  the 
system  begins  to  fill  the  database  with  temporal  in¬ 
formation.  The  system  finds  a  high  confidence  di¬ 
rect  succession  pair  and  uses  tenure  midpoint  as  a 
re-estimation  procedure.  It  then  finds  start  and  end 


Figure  5:  Precision/Recall  of  CEO  name  lists  aver¬ 
aged  over  the  test  companies. 


VFF 

FCF 

Component 

Ace. 

MRR 

Ace. 

MRR 

Direct 

Succession 

62.5% 

79% 

75% 

85.4% 

Tenure 

Midpoint 

50% 

N/A 

62.5% 

N/A 

Start  Year 

37.5% 

40.6% 

50% 

55.7% 

End  Year 

12.5% 

12.5% 

6.25% 

15.1% 

Table  3:  Synthesis  of  temporal  information  for  the 
8  companies  and  16  people  in  the  set.  The  accuracy 
and  mean  reciprocal  rank  of  two  fusion  methods  are 
compared.  FCF  performs  better  than  VFF  on  most 
tasks. 


Component 

FCM 

FCF 

Direct  Succession 

63.5% 

75% 

Start  Year 

43.8% 

50% 

End  Year 

20% 

6.25% 

Table  4:  Comparison  of  field  confidence  fusion 
(FCF)  and  maximum  field  confidence  (FCM)  meth¬ 
ods. 

field  confidence  score.  On  the  whole,  FCF  outper¬ 
forms  FCM,  though  for  end  tenure  year  synthesis 
FCM  is  more  successful. 

The  performance  for  start  and  end  year  synthesis 
was  considerably  lower  than  for  the  previous  steps, 
primarily  because  there  are  few  data  redundancies  to 
exploit.  End  year  results  were  poor  in  part  because 
for  the  CEOs  still  in  office,  all  answers  were  graded 
as  incorrect.  Additionally,  the  fact  that  start  years  are 
more  common  in  the  corpus  than  end  years  caused 
errors  in  which  the  start  year  was  returned  as  the  end 
year. 

For  re-estimating  transition  years,  both  picking 
the  start  year  of  the  succeeding  member  of  the  pair 
and  picking  a  linear  combination  of  the  confidences 
were  used.  For  the  pairs,  50%  of  the  transition  years 
are  predicted  correctly  using  the  start  year  of  the 
next  candidate,  and  25%  using  linear  interpolation. 


years  and  uses  the  start  years  and  ordering  to  esti¬ 
mate  the  transition  year.  For  the  direct  succession 
and  tenure  midpoint  reordering,  there  are  8  pairs  to 
be  evaluated,  one  for  each  company.  For  the  start 
and  end  years,  there  are  16  people  (one  pair  of  CEOs 
for  each  company). 

Table  3  and  shows  performance  for  each  target 
field  for  FCF  and  VFF.  The  results  suggest  that 
Viterbi  frequency  fusion  is  typically  less  effective 
than  field  confidence  fusion.  Only  in  one  case  (end 
year  accuracy)  did  VFF  perform  better  than  FCF. 
The  Direct  Succession  model  performed  slightly 
better  than  the  Tenure  Midpoint  model  for  order¬ 
ing.  The  tenure  midpoint  model  may  still  be  use¬ 
ful  to  provide  corroborating  evidence  for  the  direct 
succession  model. 

Table  4  shows  the  relative  performance  of  FCM 
and  FCF,  where  FCM  is  the  single  document  best 


5.3  Overall  Database  Accuracy 

In  total,  there  were  61  testing  database  records,  with 
3  fields  per  record  to  be  discovered  (CEO  name,  start 
year,  and  end  year).  The  derived  field  of  relative  or¬ 
dering  is  not  graded,  though  it  has  precise  informa¬ 
tion  which  would  be  useful  for  seeding  future  infor¬ 
mation  synthesis  systems. 

The  initial  system  returns  only  16  records,  and  so 
has  a  recall  of  26%.  The  low  recall  is  due  primar¬ 
ily  to  the  lack  of  the  information  on  the  web  for 
CEOs  from  decades  before  1990.  For  the  48  tar¬ 
gets  fields  recovered  by  the  system,  27  (56%)  are 
correct,  where  most  of  the  errors  are  for  incorrectly 
extracted  end  years.  This  grading  criterion  is  strict 
as  it  penalizes  cases  where  the  found  CEO  is  still  in 
office.  With  end  years  removed  for  CEOs  in  office, 
the  performance  in  correctly  filled  fields  is  67%. 


Query  Once 

Query  Many 

Component 

100  pgs 

1000  pgs 

100  pgs 

Direct 

Succession 

50% 

37.5% 

75% 

Tenure 

Midpoint 

37.5% 

62.5% 

62.5% 

Start  Year 

37.5 

37.5% 

50% 

End  Year 

31.25 

18.75% 

6% 

Table  5:  Allowing  each  information  synthesis  com¬ 
ponent  to  retrieve  its  own  corpus  yields  higher  per¬ 
formance  than  fixing  the  entire  corpus  at  the  start. 


5.4  Multiple  IR  Queries 

There  are  multiple  separate  corpora  downloaded 
for  different  base  queries:  one  for  company  names 
and  title  in  the  CEO  name  component  (up  to  1000 
pages  each),  distinct  corpora  for  each  ordered  pair 
of  CEOs  in  the  direct  succession  component  (100 
pages  each),  and  another  set  of  distinct  corpora  for 
individual  CEOs  for  tenure  years  as  well  as  start 
and  end  tenure  points  (100  pages  each).  In  order  to 
test  the  impact  of  these  distinct  information  retrieval 
steps,  results  were  compared  with  those  generated 
from  a  system  using  only  the  initial  corpus. 

Table  5  shows  the  relative  performance  for  single 
document  retrieval  step  of  100  or  1000  documents  as 
opposed  to  incremental  retrieval  steps  of  100  docu¬ 
ments  as  the  database  is  partially  filled.  Performing 
multiple  queries  appears  to  have  an  edge  over  down¬ 
loading  one  corpus,  especially  in  the  direct  succes¬ 
sion  and  tenure  midpoint  estimation  steps.  For  end 
tenure  year  the  larger  corpus  lead  to  better  perfor¬ 
mance,  which  suggests  a  possible  inefficiency  in  the 
retrieval  set  for  CEO  end  tenure  year. 

6  Related  Work 

The  term  “information  synthesis”  has  been  used  by 
(Blake  and  Pratt,  2002)  and  (Blake,  2005)  to  de¬ 
scribe  a  human-computer  collaborative  process  of 
retrieval,  extraction,  and  analysis  of  research  liter¬ 
ature.  Amigo  et  al.  (2004)  use  the  term  information 
synthesis  for  “topic  oriented,  multi-document  sum¬ 
marization”. 

There  has  been  relatively  little  work  on  extraction 
of  temporal  facts.  There  is  related  work  in  temporal 
summarization  by  sentence  selection  to  create  time¬ 


lines  (Allan  et  al.,  2001;  Chieu  and  Lee,  2004),  and 
TIMEX  extraction  and  resolution  (Mani  and  Wilson, 
2000).  Pustejovsky  et  al.  (2003)  describe  a  language 
for  annotating  time  events,  but  does  not  provide  a 
way  to  extract  this  information. 

Brin  (1998),  Agichtein  and  Gravano  (2000), 
and  Ravichandran  and  Hovy  (2002)  present  related 
methods  for  training  an  information  extraction  sys¬ 
tem  by  example  facts.  Alternative  types  of  training 
can  be  found  in  (Riloff,  1996),  which  trains  from 
texts  marked  relevant  and  irrelevant,  and  (Etzioni  et 
al.,  2005),  which  trains  from  single  example  patterns 
such  as  “ actor  starred  in  film”. 

Fusion  of  extracted  facts  is  a  relatively  new  area 
of  investigation.  Prior  work  in  the  area  includes 
(Skounakis  and  Craven,  2003)  and  (Downey  et  al., 
2005)  which  present  models  for  information  fusion 
for  facts  extracted  by  classifiers.  Closely  related 
are  (Finkel  et  al.,  2005)  and  (Sutton  and  McCall um, 
2004)  which  present  methods  for  the  joint  labeling 
of  named  entities  in  text  using  graphical  models  for 
single  document  extraction.  Mann  and  Yarowsky 
(2005)  introduce  two  simple  methods  for  fact  fusion 
for  sequence  models. 

The  problem  of  management  succession  has  been 
studied  in  the  context  of  the  MUC-6  evaluation  (Gr- 
ishman  and  Sundheim,  1996),  which  included  an 
evaluation  of  extraction  of  management  succession 
events  from  single  document.  Most  of  the  systems 
developed  for  this  task  were  hand-crafted,  knowl¬ 
edge  engineered  systems.  Notable  exceptions  are 
(Soderland,  1999),  which  learned  a  set  of  regular  ex¬ 
pressions,  and  (Chieu  and  Ng,  2002),  which  used 
a  log-linear  classifier.  Both  systems  extract  non¬ 
temporal  succession  events  and  find  the  company, 
the  position,  the  previous  position  holder,  and  the 
successor.  For  that  task,  Chieu  and  Ng  (2002)  re¬ 
ports  results  of  60%  F-measure  for  multi-slot  man¬ 
agement  succession  extraction  from  a  single  docu¬ 
ment  but  does  not  extract  start  and  end  years.  These 
improved  results  can  be  attributed  to  the  presence 
of  labeled  training  data  (6915  annotated  instances), 
a  more  homogeneous  corpus  made  up  of  newswire, 
and  matched  training  and  test  data.  Additionally,  the 
system  doesn’t  evaluate  on  start  and  end  year  extrac¬ 
tion,  the  targets  found  in  this  paper  to  be  the  most 
difficult. 


7  Conclusion 

This  paper  presented  a  system  for  synthesizing  time- 
bounded  facts  from  large  corpora  for  timeline  con¬ 
struction.  This  is  a  novel  information  analysis  task 
which  is  made  possible  by  minimally  supervised 
training  of  sentence  extractors,  redundant  corpora 
that  compensate  for  noisy  extraction,  and  dependen¬ 
cies  between  related  facts.  Incremental  construction 
of  databases  by  linked  information  synthesis  compo¬ 
nents  allows  for  the  gradual  aggregation  of  semantic 
networks  of  facts,  and  the  data  synthesized  in  this 
paper  could  serve  as  input  to  yet  another  processing 
step. 

An  information  synthesis  component  was  pre¬ 
sented  which  retrieved  relevant  documents,  ex¬ 
tracted  facts  from  sentences,  and  fused  the  result¬ 
ing  facts.  Field  confidence  fusion  was  shown  to  be 
an  effective  method  for  cross-document  fusion.  Re¬ 
search  into  additional  synthesis  components  which 
rely  on  information  not  present  in  a  single  sentence 
is  a  promising  area  of  future  work. 

The  resultant  timelines  provide  recent  CEO  suc¬ 
cession  information  including  relative  order,  start 
years,  and  end  years.  These  temporal  attributes  are 
fundamental  properties  of  time -bounded  facts  and 
may  be  used  for  related  synthesis  tasks. 

Please  contact  the  author  for  access  to  the  train¬ 
ing,  development,  and  test  database  of  CEO  tenure 
information. 
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